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A general analysis for the competition between breached pairing (BP) and LOFF pairing mech- 
anisms in asymmetric fermion superfluids is presented in the frame of a four fermion interaction 
model. Two physical conditions which can induce mismatched Fermi surfaces are considered: (1) 
fixed chemical potential asymmetry Sjj, and (2) fixed fermion number asymmetry a. In case (1), the 
BP state is ruled out because of Sarma instability, and the LOFF state is thermodynamically stable 
in a narrow window of Sfi. In case (2) , while the Sarma instability can be avoided and both the BP 
and LOFF states can survive provided a is less than the corresponding critical value, the BP state 
suffers magnetic instability and the LOFF state is always thermodynamically stable. While the 
LOFF window in case (2) is much larger than the one in the conventional case (1), the longitudinal 
superfluid density of the LOFF state is negative for small a and it suffers magnetic instability too. 
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I. INTRODUCTION 



The fermion pairing between different species with 
mismatched Fermi surfaces, which was discussed many 
years ago, promoted new interest in both theoretic and 
experimental studies in recent years. The mismatched 
Fermi surfaces can be realized, for instance, in a super- 
conductor in an external magnetic fieldQ or a strong 
spin-exchange field0, 0, 0, an electronic gas with two 
species of electrons from different bandsQ, a super- 
conductor with overlapping bands Q, trapped ions 
with dipolar interactions,^, a mixture of two types 
of fermionic cold atoms with different densities and/or 
masses 0, 0, 0, 0|, an isospin asymmetric nuclear 
matter with proton-neutron pairing 13J, and a neutral 
dense quark matter[ll [HElElliiEHE,!!! 
HE 1^ HE EE' In the early studies of superconduc- 
tivity, Sarma[l| found a spatially isotropic and uniform 
state where there exist gapless modes, namely the ex- 
citation of these quasiparticle modes needs no energy. 
However, this state is energetically unstable, compared 
with the fully gapped BCS state. This is the so-called 
Sarma instability [l|. Recently, this instability was widely 
discussed in general case|23, and in neutral color 
superconductivity jlE 01 ■ It is now widely accepted that 
the Sarma instability can be avoided through two pos- 
sible ways, fixed number difference of the two kinds of 
fermions [23,12^ 123,123 or a proper momentum structure 
of the attractive interaction between the two speciespTj. 
The Sarma state is now also called breached pairing (BP) 
state, since in this state the dispersion relation of one 
branch of the quasi-particles has two zero points at mo- 
menta pi and p2 where gapless excitations happen, and 
the superfluid component in the regions p < pi and 
p > P2 is breached by a normal component in the rc- 
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gion pi <p < P2. 

The LOFF state which is a spatially anisotropic ground 
state was proposed in the study of superconductors in a 
strong spin-exchange field by Larkin, Ovchinnikov, Fulde 
andFerrellfiH. In this ground state, the rotational sym- 
metry and/or the translational symmetry of the system 
are spontaneously broken. In the study of asymmetric 
nuclear matter, asymmetric atomic fermion gas and color 
superconductivity, the LOFF phase has been widely in- 
vestigated '29','33,l3l|3l|3E|3llMIM|3lH|3iliE 
El, 42,4ii4^]. 

While the Sarma instability can be avoided, a new dy- 
namical instability of the BP state which is called mag- 
netic instability in the literatures arises in the study of 
dense quark matter. It is found that the response to ex- 
ternal color magnetic field in neutral dense quark matter 
in the BP state is paramagnetic, i.e., the squared Meiss- 
ner masses of some gluons become negative |4a 147114^143. 
[HE]. This instability was also studied in atomic fermion 
system |5l|. A general analysis for asymmetric supercon- 
ductors shows that this dynamical instability is related 
to the breached pairing mechanism only and indepen- 
dent of the details of the attractive interactions [S^. It 
is recently pointed out that, due to the magnetic insta- 
bility, the free energy of the LOFF state or some other 
exotic anisotropic states is probably lower than that of 
the BP state[5i|5E|5l|5E|5i,|5l|5E|5Ei3. However, 
this conclusion is obtained from the study for asymmetric 
systems with fixed chemical potentials. In many physi- 
cal cases we are interested in, the chemical potentials 
are not directly fixed, instead the particle numbers are 
fixed or some other constraints on the particle numbers 
are required. For these systems, the magnetic instabil- 
ity is still not clear. In this paper we will give a general 
and systematic comparison between the BP and LOFF 
states in asymmetric fermion superfluids under different 
conditions which can lead to mismatched Fermi surfaces. 

The paper is organized as follows. In Section ^1 wc 
briefly review the formalism for the isotropic BCS, BP 
and anisotropic LOFF states. In Section ITTll we analyti- 
cally discuss the thermodynamic and dynamical instabil- 
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ities of the BP state, and show why the LOFF state may 
be more favored than the BP state. In Sections IIVI and 
fvl we discuss, respectively, the systems with fixed chem- 
ical potential asymmetry and fixed density asymmetry. 
We summarize in Section IVTl We use the natural unit of 
c = h = I through the paper. 



LOFF state with q ^ Q. After a transformation of the 
fermion fields Xa{x) = e^^'^^pa{x) and Xb{x) = e^^'^tpbix), 
we can directly evaluate the Gaussian path integral and 
obtain the thermodynamic potential 



9 



(2^ 



Trlng-i(zw„,p) (8) 



II. MODEL AND FORMALISM 

To have a general investigation, we consider the model 
Lagrangian with two species of fermions interacting with 
each other via a point interaction. 



d_ 



2rai 



i—a,h 



tpiix) 



(1) 



where ipaix), 4'bix) are fermion fields for the two species 
a and b with space-time x = (r, x), the coupling constant 
g is positive to keep the interaction attractive, and 
nib are the masses of the two species, and /ia and /i^ are 
the chemical potentials. 

The key quantity to describe a thermodynamic system 
is the partition function which can be defined as 



Z = / [dVa] m] [d^b] I d^SC 



(2) 



in the imaginary time (r) formalism of finite temperature 
field theory. According to the standard BCS theory, we 
introduce the order parameter field <i>(.T) and its complex 
conjugate 

$(x) = g{Mx)Mx)) , 'i>*{x) = g{ra{xWb{x)) , (3) 

where the symbol ( ) means ensemble average. Introduc- 
ing the Nambu-Gorkov space defined as 



*(a;) 



i'aix) 

rbix) 



■^*{x) = {ra{x) Mx)) , (4) 



the partition function in mean field approximation can 
be written as 

Zmf = I [dnd^*]J'''''f d^s{^g-^^+mVg) 
with the inverse of the mean field fermion propagator 
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To have a unified approach for the isotropic BCS, BP 
states and anisotropic LOFF state, we take in the follow- 
ing a specific ansatz for the order parameter fields 



2iq-x 



(7) 



Here A is a real quantity. Obviously, we recover the 
isotropic state with q = Q and obtain the anisotropic 



in terms of the inverse fermion propagator in the momen- 
tum and frequency space 



lUJ. 
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iuJn + 



(9) 



Ha and eb 



with (a - 2m,. f^i^ ""^ - 2m. 

form of the propagator can be written as 



Hb- The explicit 



y(lU;n,P)- (^g2l(*C.„,p) g22{lUJn,p) 

with the elements 
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C/ii(iw„,p) = 
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Gl2iil^n,P) = 




-A 




{iuJn 
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Q2l{ii^n,P) = 




-A 




{iuJn 
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(11) 



where the quantities e+ , e_ and eA are defined as 

Efi + Eb — Eb 
= 7^ ' ^- = n ' 

EA = 



ei + A2. 



(12) 



Using the identity Trln = Indet, the thermodynamic 
potential can be evaluated as 



A2 

r!(A,g) = 

9 



(2^ 



^+Tln(l + e-^) 



+ (^+rin(l + e-^))-e+], (13) 

where ea and es are the quasiparticle energies 

eA = e-+eA, es = £- - eA . (14) 

The gap equation which determines the order param- 
eter A self-consistently is related to the off-diagonal ele- 
ment of the propagator matrix. 



d^p 
(2^ 



^12(it^n,p), 



(15) 



which is equivalent to the stationary condition of the 
thermodynamic potential, dil/dA = 0. After the sum- 
mation over the fermion frequency, it becomes 



A(i - giA) = 0, 



(16) 



3 



where the function I a is defined as 



(27r)3 



2eA 



(17) 



with the Fcrmi-Dirac distribution function /(x) = 
l/(e=^/'^ + 1). The physical pair momentum q is de- 
termined by minimizing the thermodynamic potential. 
In the following, we choose a suitable frame where the 
z axis is along the direction of the pair momentum, 
q = (0,0, g). Calculating the first order derivative of 
the thermodynamic potential with respect to q, the gap 
equation dV,/dq = for g is explicitly written as 



(2^ 
1 



(/(-eB)-/(eA))^ 



Ji + (/(eA) + /(-es))^ 



.(18) 



It is easy to see that g = is a trivial solution of the 
equation, which corresponds to the isotropic BCS or BP 

state. 

The occupation numbers are determined through the 
diagonal elements of the fermion propagator, 

naip) = Tlim Ven(«a;„,p)e'""^ 

r/— J-0 ^ — ^ 

n 

nbip) = -TlimY^g22{iuJn,p)e-^-''. (19) 

n 

Completing the Matsubara frequency summation, we ob- 
tain 



Ubip) = ulf{-eB)+vlf{-eA) 



(20) 



4 = {l + e+/eA)/2 and = 
(1 — e+/eA) /2. At zero temperature, they are reduced 
to 

naip) = vpicA - e_) + ul6{-€A - e_), 

ribip) = vle{eA + e-)+ule{-eA + e-). (21) 

The particle number densities rio and rib can be calcu- 
lated from 

(lifs^^C^' = j -^nb{p). (22) 

For a free Fermi gas, we can define the Fermi momenta 
■p"p and p\r for the two species, and the particle number 
densities are then expressed as 



Ml! 

67r2 ' 



67r2 ■ 



nb 



(23) 



For convenience, we define total Fermi momentum pp. 
Fermi energy ep and Fermi velocity vp through the total 



number density n = Ha + Ub and the reduced mass m = 
2mamb/{ma + nib), n = p|,/(37r2), eF = p|/(2m) and 
vf = PF/m,. 

The above formalism is general for any system with 
Ma 7^ Mb and TTia mb- In the following we will consider 
only the case without mass difference between the two 
species, rua = mb = m, which is relevant for the study of 
cold atomic fermion gas, isospin asymmetric nuclear mat- 
ter, and two flavor color superconducting quark matter. 
For convenience, we define the average of the chemical 
potentials fia and /Xb and their difference, 



tJ'a+ Ub . Mb - Ma 

M = 7, ' '^M= 7, • 



(24) 



In the weak coupling BCS region, the chemical potentials 
Ha and /if, are approximately the corresponding Fermi 
energies e% = {p%)'^/{2m) and = (p^)2/(2m), and 
the pairing gap A and the Fermi surface mismatch SjjL 
are much smaller than the average chemical potential, 
A ~ «: M- 



A. Isotropic Superfluid with q = Q 

We discuss firstly the possible isotropic superfluid 
state, i.e., the case with q = 0. The dispersion relations 
of the quasiparticle excitations are expressed as 




(25) 



Without losing generality, we set > 0. It is easy to 
see from the equation = that only in the case with 
A < there are gapless excitation at momenta 



Pi 



^2771 (^/i - 



m'-A2 , 



P2 = J 2m (/i+VV-A^). 



(26) 



In the gapless state, we have na{p) = nbip) = for p < 
pi and p> P2 and Uaip) = 0, Ubip) = 1 ior pi < p < p2, 
the superfluidity state with fermion pairing in the regions 
p < Pi and p > P2 is breached by the normal fermion gas 
in the region pi < p < p2. This is the reason why people 
call the gapless state as breached pairing (BP) state. In 
the case with A > there is no gapless excitation, and 
the system is in BCS phase with naip) = f^bip) = Vp for 
all p. 



B. Anisotropic Superfluid with q^O 

We now discuss the anisotropic superfluid state with 
qy^O. The dispersion relations of the elementary excita- 
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tions can be written as 



€A = 5iJL-\ cos 

m 




2m 



= ., + ^cos._^(^-,J +A^(27) 

where 9 is the angle between the momenta p and q. In 
many of the previous investigations, the term /2m 
was neglected and the term pq cos 9 /m was approxi- 
mated by wf9COS00,|3,H since q/pF <C 1 in weak cou- 
pling limit. Taking this approximation, in the case of 
Se = S/jL + VFqcos9 > A, the gapless excitation happens 
at 



Pi{9) = J2m{^i-JSI-A^ 



P2{9) = ^ 2m (^Ai + 



- A2 



(28) 



For the following numerical research, we will do full cal- 
culation without employing this approximation. We will 
see that the anisotropic LOFF state possesses always un- 
equal particle numbers for the two species, and hence the 
LOFF superfluid is gapless. 

Since the gap equation for A suffers ultraviolet diver- 
gence, a regularization scheme is needed. We consider 
now two regularization schemes. In the first scheme, we 
simply employ a hard cutoff pA for the three momentum. 
Such a regularization scheme is related to the study of 
color superconductivity at moderate baryon density and 
isospin asymmetric nuclear matter. To avoid a specific 
choice of model parameters, such as the mass to, we ex- 
press the coupling g in terms of the s-wave scattering 
length as as g — with a — \as\. Once this is done, 
the two original parameters pA and g in the model are 
replaced by two dimcnsionless parameters A = (pa/pf)'^ 
and pFCi- In the weak coupling BCS region < pFO, < 1 
and for not very large cutoff 1 < A < 2, the solution of 
the gap equation can be well described by 



Ao 



Va2 



1 e 2ppa 



(29) 



in the symmetric case with Sfi — 0. 

In the second regularization scheme, we replace the 
bare coupling g by the low energy limit of the two-body 
T-matrixl6ll 



47ra, 



1 
9 



d^p 



(27r)3 p2 



(30) 



Such a regularization scheme is often used in the study 
of atomic fermion gas. In this scheme, it is not necessary 
to introduce a momentum cutoff, and all physical results 
depend only on the dimcnsionless coupling p^^a. This can 
be seen explicitly by scaling all the quantities with energy 



dimension by the Fermi energy and all the quantities 
with momentum dimension by the Fermi momentum pf, 



PF J \PF , 



Va = , l^b = : 



v=—, 5v~—, 5— — , t— — . (31) 
€f eF ef ef 

With these dimcnsionless quantities, we obtain the scaled 
gap equations. 
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1 I— I 1 fi^z,x) fi^z.x) 

dz\J z / ax — ' — 

-1 v^(z -^Zq- + ppa 



= / dz\fz I dx 



+ 



z + z„ - V 



^{Z + Zq- Vf + (52 

the scaled thermodynamic potential 
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dz\fz 



dx 



tln(l + e" 



|^-|^+nn(l-|-e-^)J -{z^Zq~v) 



2pFa 

and the scaled number densities, 



(33) 



no 



1 



dz^z / dx [ulf{el,) -f vlfiel^)] , (34) 



^ = \jdz^zf dx [1 - ulfiel,) - vlfiei,)] , 
no Z J J_i 

where the normalization constants uq and Eq are chosen 
to be no = p^/{ATr'^) and Eq ~ pp/{8TT'^m), the quasi- 
particle energies and the functions m, v are reexpressed in 
terms of the dimcnsionless quantities. 



^ = Sv + 2x^ZZq + y [z + Zq - v) +(52, (35) 



ef 3, = 5v + 2x^fzz'q — \J {z + Zq — vf' + (52, 



-[1 + {Z + Zq- V)U [Z + Zq- vf + (52 



[l-{z + Zq- V)/^{Z + Zq-VY +6'^^ . 



V,_= -[1-{Z + Zq 

In weak coupling limit, the solution of the gap equation 
is 



Ao ^ -irepe ^"f 



(36) 



in the symmetric case with Sfi — 0. It is obvious that 
if we take A — \/2, the results in the two regulariza- 
tion schemes are the same in symmetric case and will 
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not make significant difference in asymmetric case in the 
region < ppa < 1. 

While the values of A, and q depend on the cut- 
off A, we emphasis that the scaled quantities such as 
A/Ao,(5/i/Ao and vpl/SfJ. are regularization scheme in- 
dependent in the weak coupling region SET 



III. THERMODYNAMIC AND DYNAMICAL 
INSTABILITIES IN BP SUPERFLUID 

In this section, we discuss some possible thermody- 
namic and dynamical instabilities of the BP state. From 
the analytic analysis, we will see that there are two types 
of instabilities in the BP phase. One is the Sarma insta- 
bility which was found many years ago, and the other is 
the magnetic instability which was recently found in the 
study of gapless superconductivity. 

We first discuss the thermodynamic instability induced 
by condensate fluctuations. To this end, we expand the 
thermodynamic potential in powers of an infinitely small 
fiuctuation (5, 



1 d^fl 



(37) 



The linear term on the r.h.s. vanishes automatically due 
to the gap equation for A. At zero temperature, the gap 
susceptibility can be evaluated as 



9A2 



dpp'^ A 



2 r 



^(eA - Sn) 



CA 



In weak coupling region with A 
approximated as 



(38) 
- (5(eA - Sfj.) . 
6fi <^ /I, it can be 



KA 



mpF 



Sfi0{6fi - A) 
^dn^ - A2 



(39) 



For the BCS state which satisfies the constraint A > 
(5/Lt, the 5-function in H38|) disappears automatically, ka 
is obviously positive, and the state is stable. For the 
BP state with A < 5^, the integral over the (5-function 
becomes nonzero and dominates the susceptibility. In 
this case, we have ka < 0, and the state is unstable. 
This is the so-called Sarma instability. 

For the systems with fixed chemical potentials /ia and 
fib or fixed total chemical potential /i and chemical po- 
tential difference 6fi, we can directly compare Sfi with 
A and see if the state is stable. However, for a sys- 
tems under some constraints on particle numbers, the 
grand canonical ensemble should be replaced by canoni- 
cal ensemble, and the essential quantity to describe the 
thermodynamics of the systems is the free energy F in- 
stead of the thermodynamic potential fl. In this case, 
the chemical potentials are no longer independent ther- 
modynamic variables, A and Sfi should be determined 



self-consistently via solving the coupled set of equations 
for the gap parameter and the constraints on the particle 
numbers. Therefore, it is not convenient to judge the in- 
stability of the BP state through ka- On the other hand, 
when the number difference between the two species is 
fixed, the BCS solution is ruled out, and only the BP 
state survives pol 121) . A direct way for the instability 
analysis is to compare the thermodynamic potentials or 
free energies themselves in BP and LOFF states, and see 
which one is more stable. 

We now turn to discuss the instability induced by a 
small superfiuid velocity Vs- When the superfiuid moves 
with a uniform velocity Vs, the condensate transforms 

like $ ^ q>e2zmv, x g^^^ ^*g-2imff,.x^ ^^le su- 

percurrent js and the superfiuid density ps are defined 
via the thermodynamic potential 



fl{Vs) = n{Vs = 0) + js ■ Vs + -j^PsVs 



(40) 



When the density Ps is negative, the superfiuid is dy- 
namically unstable. This dynamical instability can be 
related to the thermodynamic instability induced by pair 
momentum fiuctuations. We perturb the BP state with 
an infinitely small LOFF momentum q and expand the 
thermodynamic potential around g = 0, 



n{A,q)^n{A,Q) ^ — 



q=0 



q 



(41) 



q=0 



The linear term vanishes due to the gap equation for 
the pair momentum, and we can easily observe the rela- 
tions between the superfiuid density Psjl^l) the Meiss- 
|52l | and the pair momentum 



ner mass s quar ed 
susceptibility [53 

Ps = 



d^n/dq^ 



(42) 



Using the result in [5 at zero temperature we have 

Ps = mn--—r dpp'^S{€A - Sfi). (43) 
67^2 Jo 

In the weak coupling region, sa^S^ — has two possible 
roots pi and p2, and ps is reduced to 



Ps 



1-77 



Sfi9{dp - A) 



(44) 



with the coefficient r/ — (pf + pf) / {Gtt^ti). Since the 
total fermion density n at T = can be expressed as 



Pi 



1 



67r2 



dpp'u^ + / dpp'v'p , (45) 



and the two integrals in the bracket are very small in 
weak coupling region and thus can be neglectedlsj, the 
coefficient 77 is approximately equal to 1, and ps can be 
expressed as 



1 



5fie{5p - A) 
^jSfjfl - A2 



(46) 
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It is easy to see that in the BP phase with A < 5^, ps 
becomes negative. This is the so called magnetic insta- 
bility, since it is directly related to the negative Meissner 
mass squared, if the fermions are charged. This dynam- 
ical instability implies a lower free energy of the LOFF 
state in comparison with the BP state. 

As we mentioned above, however, for the systems with 
fixed density asymmetry instead of chemical potential 
asymmetry, the essential thermodynamic function is the 
free energy F instead of 17. For such systems, the above 
argument fails and we need to calculate the free energy 
directly and then determine the thermodynamically sta- 
ble state. 

We end this section with a qualitative discussion on the 
instability analysis in strong coupling case. For such sys- 
tems, the condition fi » does not hold and even 
/i < happens in the BEC region, the node pi be- 
comes imaginary, and there is only one gapless node 
P2. In this case, it has been argued that the gapless 
state is automatically free from the Sarma and magnetic 
instabilities!^ [b^, and therefore, the LOFF state be- 
comes less favored than the BP state. In fact, some re- 
cent studies have shown that the BP state will be the 
ground state of asymmetric fermion gas when the cou- 
pling is stronglHlHlille^. 



IV. FIXED CHEMICAL POTENTIAL 
ASYMMETRY 

We now solve the gap equations to find the thermody- 
namically stable state among the normal, BCS, BP and 
LOFF states. The four states are defined through the 
solutions of the gap equations, 

(1) normal phase with A = 0, 

(2) BCS phase with A 7^ 0, g = and A > dp, 

(3) BP phase with A 7^ 0, g = and A < (5/i, 

(4) L0FF phase with A 7^ 0, g 7^ 0. 

In this section we consider the case of fixed chemi- 
cal potentials fia and fib with /ia 7^ fib- For example, if 
a superfluid with cooper pair composed of spin up and 
down fermions is placed in an external magnetic field H , 
there will be an effective chemical potential asymmetry 
6fi = fisH with the fermion magneton fiB- When the 
external field is fixed, the chemical potential asymme- 
try is naturally fixed. In this case, the system is con- 
nected to reservoirs of each species and the particle num- 
bers of the two species are not conserved. Fig^ shows 
the condensate A, scaled by its value Aq in the corre- 
sponding symmetric system with 5fL = 0, as a function 
of the chemical potential difference 5fi scaled by Aq too 
at ppa = 0.4 and T — Q. In a wide range of coupling 
< ppa < 1, we found that the results are almost the 
same for the scaled quantities. We compare first the 
three uniform phases. The solution A = is the trivial 
solution of the gap equations which describes the nor- 
mal phase without superfluidity, the BCS solution does 
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FIG. 1: The scaled gap parameter A/Ao as a function of the 
scaled chemical potential difference Sfi/ Aq at ppa — 0.4 and 
T = for normal, BCS, BP and LOFF (including LOFFl by 
dashed line and LOFF2 by solid line) states. The thick lines 
indicate the real ground states in different regions. 



not change with increasing dfj, and keeps its symmetric 
value until Sfi = Aq, and the BP solution starts to ap- 
pear at dfi = O.5A0 and satisfies A < Sfi in the region 
O.5A0 < Sfi < Aq. T he BP solution can be expressed 
analytically as A = •\/Ao(2^/i~^"AoyP| . From the anal- 
ysis in Section IIIII the BP state is thermodynamically 
unstable. This can be seen clearly from the thermody- 
namic potential as a function of the scaled condensate 
A/Ag for three values of scaled chemical potential differ- 
ence Sfi/Ao, shown in Fig|21 For Sfi < 0.5Ao, there are 
a maximum at A = and a minimum at A = Aq, cor- 
responding, respectively, to the normal phase and BCS 
phase, and for O.SAq < Sfi < 0.707Ao ~ A0/V2, there 
are a maximum at < A < Aq, a local minimum at 
A = and the lowest minimum at A = Aq, correspond- 
ing, respectively, to the BP state, normal state and BCS 
state. For Sfi > Aq/V^, the maximum is still located 
at < A < Aq, but the local minimum and the low- 
est minimum exchange positions. Therefore, among the 
three uniform phases the BCS phase is stable in the re- 
gion < < Aq/^/2 and the normal phase becomes 
stable for Sfi > Aa/y/2. 

We now include the anisotropic LOFF state. The 
scaled LOFF gap A/Aq and scaled LOFF momentum 
vpq/Sfi are shown in FigsQ] and 13 as functions of the 
scaled chemical potential difference Sfi/ Aq at ppa = 0.4 
and T = 0. The LOFF state starts at Sfi = 0.635Ao and 
there are two branches called LOFFl and L0FF2. To 
find the true ground state, we compared the thermody- 
namic potentials for the four phases in Fig^ It is easy 
to see that, below Sfii ~ Aq/V^ = 0.707Ao the LOFFl 
and L0FF2 are unstable and the system is in BCS state, 
above Sfj,2 — 0.754Ao the normal state becomes stable, 
and in between Sfii and Sfi2 the L0FF2 state is the true 
ground state. In this LOFF window, the gap A starts 
from 0.24Ao at Sfi = Sfii and decreases monotonically 
to at Sfi — 5fi2, the scaled LOFF monentum vpq/Sfi 
is almost a constant, 1.2 < vpq/Sfi < 1.3 in the win- 
dow. The critical values Sfii and Sfi2 for the LOFF state 
agree with the analytical and numerical results obtained 



5^/Ao=0.6 



A/Ao 



ppa 



FIG. 2: The scaled thermodynamic potential S7/i5o as a func- 
tion of the scaled gap parameter A/Aq at ppa = 0.4, T = 
and q — for three chemical potential values, < 5/i/Ao < 
1/2,1/2 < Sfi/Ao < and Sfi/ Aq > 1/^2. We have 

chosen the thermodynamic potential of the BCS state to be 
zero. 



L0FF2 ^ 






L0FF1~\ 



0.2 0.4 0.6 O.B 1 



FIG. 3: The scaled LOFF momentum vpq/S^ as a function of 
the scaled chemical potential difference Sjj,/ Aq at ppa = 0.4 
and r = for LOFFl and L0FF2. 



BP ^_,„.----''''''''^ 
LOFFJ, 






- ^ LbFF2 







0.64 0,66 0.68 0.7 0.72 0.74 0.76 0.7i 



iS^z/Ao 

FIG. 4: The scaled thermodynamic potential S7/i5o as a func- 
tion of the scaled chemical potential difference iS/i/Ao at 
ppa = 0.4 and T = for normal, BCS, BP, LOFFl, and 
LOFF2 states. We have chosen the thermodynamic potential 
of the normal state to be zero. 



in other studies in weak coupling limit 13. Il. l2^l33 . l5^ . 
In FiglSlwe showed J^i/Aq and 5^2/ Ao as functions of 
ypa. In small coupling region, they are almost constants 
and agree well with the analytical result, while when the 
coupling is strong enough, the LOFF window becomes 



FIG. 5: The starting and ending chemical potential difference 
(5/ii/Ao and 5^2/ Aq for the LOFF window as functions of 
ppa. 



small. This behavior agrees with the result in crystalline 
color superconductivity [2^. 

We investigated also systems under condition of fixed 
total number n and chemical potential asymmetry djj,. 
The behavior of the gap parameter, pair momentum and 
the free energy F = -\- is almost the same as what 
we showed here. The reason may be the assumption 
rua — m,}, considered in this paper. With a large mass 
difference, the two cases will be quite different |6^. 

V. FIXED DENSITY ASYMMETRY 

For many asymmetric fermion systems, the fixed quan- 
tity is the number density asymmetry instead of the 
chemical potential asymmetry. For example, in cold atom 
gas of ^Li the number of the atoms in each hyperfine 
state can be directly adjusted, in isospin asymmetric nu- 
clear matter the density difference between neutron and 
proton is fixed, and in two flavor dense quark matter, 
the density difference between u and d quarks is required 
to satisfy charge neutrality. In this section, we assume 
that the number densities of the two species are fixed. 
In this case, the chemical potentials fia and are not 
thermodynamic variables, they should be determined by 
the known number densities Ua and Ub- As a result, the 
essential quantity to describe the thermodynamics of the 
system is the free energy defined as 

F = CI + fiaUa + ntnt- (47) 

In general case of Ha ^ Uf,, instead of Ha and nt, we 
can equivalently describe the system by the total number 
n = Ua + rib and the quantity 



rib + ria 

which controls the degree of asymmetry between the two 
species and satisfies < a < 1. Obviously, once Ua ^ Ub 
or a ^ 0, the BCS pairing mechanism is ruled out, only 
the normal phase, BP phase and LOFF phase are left as 
candidates of ground states. 
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FIG. 6: The scaled gap parameter A/Ao and the chemical 
potential difference 5fi/Ao as functions of the scaled degree 
of asymmetry a/ac at T = 0. ac is the critical asymmetry 
for the BP superfluidity. 



FIG. 7: The scaled free energy F/Eo as a function of the 
scaled gap parameter A/Ao at T = 0,a = 0.05 and ppa = 
0.8. The minimum and maximum correspond, respectively, 
to the BP and normal states. 



Considering the coupled set of equations for the gap 
parameter and the particle numbers 



1 



1 



(49) 



the gap parameter A and chemical potential difference 
Sfj, in the BP state as functions of the asymmetry degree 
a are shown in Fig|Slat T = 0, where the scaling param- 
eter ac is the critical asymmetry where the BP superflu- 
idity disappears. While both A and Sfi drop down with 
increasing degree of asymmetry, they always satisfy the 
condition for the BP state, A < (5/i, and the difference 
between them becomes larger and larger. Finally, they 
end at the critical value ac- From the comparison with 
Fig^ where Sfi is fixed, the BP gap parameter here still 
starts with A = at (5/x = 0.5Ao and ends with A = Aq 
at 6^ ~ Aq. The a-dependence of A and dfu, and the 
critical value ac in our calculation agree well with the 
analytic results obtained in weak coupling region f65ll66| . 



a(q) _ r~a^ 

Ao V ac' 



M^ = l-1^. (50) 
Ao 2ac ^ ' 



The question is which of the normal and BP states is 
relatively stable. From the direct calculation of the free 
energy as a function of the gap parameter for a specific 
asymmetry a = 0.05 and coupling ppa = 0.8 at T = 0, 
shown in Figd the trivial solution A = is the maximum 
but the BP solution A 7^ is the minimum of the free 
energy. We have checked this result for other values of 
a and ppa, the qualitative behavior does not change. 
Therefore, the Sarma instability is avoided and the BP 
state becomes a stable solution in the family of isotropic 
phases. It is necessary to emphasis that the point to 
avoid the Sarma instability is the constraint of unequal 
numbers of the two species. 

The LOFF solution is obtained through solving the 
equations for the gap A and LOFF momentum, together 
with the known particle number densities Ua and rif,. At 
zero temperature, the scaled gap parameter and LOFF 
momentum are plotted as functions of the scaled degree 













^\^L0FF2 







FIG. 8: The scaled LOFF gap A/Ao and chemical potential 
difference Sfi/ Ao as functions of the scaled degree of asym- 
metry a/ac at T = 0. The dashed and solid lines correspond, 
respectively, to the branches LOFFl and LOFF2. 



of asymmetry a/ac in Figs|Sl and |^ Here ac is the 
critical asymmetry where the LOFF superfluidity dis- 
appears. We have checked that for any a < ac, the 
LOFF solution obtained here is thermodynamically sta- 
ble against a small perturbation of A, i.e., it corresponds 
to the minimum of _F as a function of A. Exactly the 
same as that shown in Fig^ the two branches LOFFl 
and L0FF2 meet at Sn = 0.635Ao, LOFFl ends at 
A = Ao and Sfi = Aq, and L0FF2 ends at A = and 
6fi ~ O.754A0. If we denote the asymmetry correspond- 
ing to the meeting point of LOFFl and L0FF2 as ao, 
we found ao — 0.55ac- The scaled LOFF momentum 
can vary in the range < vpq/6fj, < 1.3. In Fig llOl we 
showed the critical asymmetry for BP and LOFF states 
as a function of coupling strength ppa. We found that 
the critical asymmetry of LOFF state is larger than that 
of BP state, and the difference between the two increases 
with increasing coupling strength. It is necessary to note 
that the condition for the BP gap parameter, A < 5^, 
is no longer necessary for the LOFF gap parameter, be- 
cause of the term pq cos 9 /m in the dispersion relation 

Now the question left is to extract the real ground 
state from the isotropic and anisotropic phases. Since 
the BCS phase is already kicked out, we need to com- 



/ L0FF1 

/ 

0.2 . / 



FIG. 9: The scaled LOFF momentum vpq/Sfi as a func- 
tion of the scaled degree of asymmetry a/ac a,t T — 0. 
The dashed and solid lines correspond, respectively, to the 
branches LOFFl and L0FF2. 



0.25 




ppa 



FIG. 10: The critical asymmetry ac as a function of the inter- 
action strength ppa. The solid and dashed lines correspond, 
respectively, to the LOFF and BP states. 



pare the free energies only for the normal, BP and LOFF 
states. Fig^J shows the free energy as a function of 
the pairing momentum at fixed degree of asymmetry and 
coupling strength. It is clear that the BP state which 
has a zero pair momentum is a maximum of the free en- 
ergy in the q direction, while the LOFF state which has 
a finite pair momentum is the minimum of the free en- 



0.02 0.04 0.06 0.01 

a 




0.025 0.05 0.075 0.1 0.125 0.15 

a 



FIG. 12: The scaled free energies for the normal, BP and 
LOFF states as functions of the degree of asymmetry a at 
T = and ppa — 0.6 and 0.8. We have chosen the free 
energy of the normal state to be zero. 




'0 0.2 0.4 0.6 0.8 1 



A/Ao 

FIG. 13: The logitudinal superfluid density, scaled by mn, as 
a function of A/Aq. 



LU 




q/PF 

FIG. 11: The scaled free energy F/Eq as a function of the 
scaled pair momentum q/pr at T = 0,a = 0.05 and ppa = 
0.8. The maximum at zero pair momentum and the minimum 
at finite pair momentum correspond, respectively, to the BP 
and LOFF states. 



ergy. We note that, the LOFF minimum in the figure 
locates at the LOFFl branch which is ruled out in the 
conventional case studied in Section Hvl For other val- 
ues of asymmetry and coupling, the F — q curve behaves 
similarly. In Fig^J we calculated the free energies for 
normal, BP and LOFF states as functions of a at two 
values of the coupling strength, ppa — 0.6 and 0.8. For 
other values of coupling in the region < ppo. < 1 we 
have checked that the qualitative behavior is the same. 
Thus we have the conclusion that, in the region with 
small asymmetry where the BP and LOFF states coex- 
ist, the LOFF is the stable one, and in the region with 
higher asymmetry where the BP disappears, the LOFF 
becomes the only possible mechanism for superfluidity. 
It is necessary to point out that, while only a part of 
L0FF2 is thermodynamically stable in the case with 
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fixed chemical potential asymmetry Sfj,, both LOFFl and 
L0FF2 are thermodynamically stable in the case with 
fixed density asymmetry a. The conventional LOFF win- 
dow 0.707Ao < Sfj,< O.754A0 corresponds to the highly 
asymmetric region 0.93ac < a < ac- If the density asym- 
metry is smaller than 0.93q!c, the LOFF solution will 
not locate in the conventional LOFF window. In a re- 
cent study of two flavor color superconductivity jH^, it is 
found that the LOFF solution satisfying electric charge 
neutrality is located at the LOFFl branch. 

We have shown that the LOFF state is thermodynam- 
ically stable in the whole region < a < ac, where the 
gap can vary in the region < A < Aq and the pair mo- 
mentum can vary in the region < vpq < ISSfi. Now 
we turn to check whether the LOFF state is dynamically 
stable in the whole region. Since the rotational symme- 
try 0(3) is broken down to 0{2), the superfluid density 
becomes a tensor pij. We decompose pij into a trans- 
verse and a longitudinal part due to the residue symme- 
try 0(2), 



Ptj = PriSij - qiqj) + PLqiq^ 



(51) 



with q = q/\q\. Neglecting the terms proportional to 
q/pF and {q/pp)"^ and replacing cos (?/m by vpqcosO 
under the assumption of small q, we get at zero temper- 
ature 



Pt = mn 



PL = rnn 



1 - - 



d cos 9 sin Orji 



1 — ^ / d cos 9 cos^ 6t]( 



m\se\-^) 



with rjg defined as 



m = 



pU O)+Pm 



(52) 



(53) 



Note that the expression pij is the same as the Meissner 
mass tensor for the 8th gluon in two flavor LOFF color 
superconductor 54] except for a pre-factor mn, if we em- 
ploy the good approximation 779 ~ 1. Using the numerical 
result for the LOFF solution, we calculated px and pL 
and found that px is almost zero, which is consistent with 
the analytical result in. 5.4] • The longitudinal superfluid 
density pL scaled by mn is shown in Fig ll3l as a function 
of the scaled gap parameter, it is positive in the region 
< A < 0.83Ao and negative for 0.83Ao < A < Aq. 
Hence, the LOFF state with 0.83 < A/Aq < 1, i.e., 
in the small asymmetry region < a < 0.2ac or small 
LOFF momentum region < vpg/Sfj, < 0.4, is dynam- 
ically unstable. For the BP state, the superfluid den- 
sity is absolutely negative due to the fact that the man- 
ifold of the gapless excitations in momentum space is 
very large. This negative superfluid density is generally 
cured in the LOFF state since the manifold of gapless 
excitations is suppressed by anisotropy. However, in the 
region 0.83 < A/Aq < 1, the scaled LOFF momentum 



vpq/Sp which reflects the degree of anisotropy becomes 
very small, the superfluid density will still be negative. 
When A/Aq — > 1, vpg/Sp, 0, the anisotropy disap- 
pears, and the superfluid density becomes divergent. 



VI. SUMMARY 

The key theoretic problem in the study of asymmetric 
fermion superfluid is the pairing mechanism. The famil- 
iar mechanisms include isotropic BCS and BP states and 
anisotropic LOFF state. The question is which of them 
is the real ground state for a system under some physical 
condition which can lead to mismatched Fermi surfaces 
of the two fermions. 

We have investigated the competition between the nor- 
mal, BCS, BP and LOFF phases in asymmetric fermion 
superfluids in the frame of a general four fermion inter- 
action model in weak coupling region. The masses of the 
two species ma and mi, are set to be equal, but their 
chemical potentials pa and pb can be different. Two con- 
straints on the system are discussed: (1) fixed chemical 
potential asymmetry and (2) fixed fermion number asym- 
metry. 

In BP phase, there are two kinds of instabilities, one is 
the Sarma instability which was found many years ago, 
and the other is the magnetic instability which was re- 
cently pointed out in the study of interior gap super- 
fluidity and gapless color superconductivity. While the 
magnetic instability gives us a strong hint that the LOFF 
phase may be stabler than the BP phase, to conflrm this 
statement needs the comparison of the free energies for 
the normal, BCS, BP and LOFF states, when the chem- 
ical potentials of the two species are not directly fixed. 
By solving the gap equations, we obtained all possible 
solutions, and then by comparing the thermodynamic 
potentials or free energies F for the four phases, we 
determined the real ground state which corresponds to 
the lowest minimum of fl or F. For case (1), the BP 
state is always unstable due to the Sarma instability, the 
system is in BCS phase when the chemical potential dif- 
ference is small and in normal phase when the difference 
is large enough, and the LOFF state can exist only in a 
narrow window of asymmetry. For case (2), the Sarma 
instability of the BP state can be avoided due to the dis- 
appearance of the BCS phase, if tIq and ni, are set to be 
unequal. However, due to the magnetic instability of the 
BP state, in the coexistence region of BP and LOFF, the 
LOFF state is always the stable one. In addition, the 
LOFF state can even survive when the BP disappears 
in the highly asymmetric region. We emphasis that the 
LOFF windows in case (1) and (2) are quite different. 
In case (1), the LOFF state is thermodynamically stable 
only in a narrow window, i.e., 0.707Ao < 6p < 0.754Ao 
in weak coupling. In this window, the gap A can vary 
in the region < A < 0.24Ao and the LOFF mo- 
mentum is almost a constant, 1.2 < vpq/Sfi < 1.3. 
However, in case (2), the LOFF state is thermodynam- 
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ically stable in a large region < a < ac, which cor- 
responds to 0.635Ao < < Aq, < a < Aq and 
< vpq < l.3Sfi. Only in the small a region corre- 
sponding to 0.83Ao < A < Aq the superfluid density is 
negative, which means that the LOFF state in this region 
is dynamically unstable. 

We considered in this paper only the asymmetric sys- 
tems with equal masses of the two kinds of fermions. 
However, a system with mass difference is of great inter- 
est, since it can be realized in the atomic fermion gas with 
a mixture of different fermionic atoms such as ^Li and 
'^^fcQ. As we have shown in the breached pairing 
state can be free from magnetic instability when the mass 
ratio of the two species becomes very large. The true 
ground state of these systems is still unknown and needs 
further investigation. The superfluidity we considered 
here is non-relativistic, which can be applied to the study 



of electronic system, atomic fermion gas, and nuclear 
matter, and can be extended to relativistic fermion su- 
perfluids if the broken symmetry is Abelian. In the study 
of color superconductivity where the broken color sym- 
metry is non- Abelian, the problem of (chromo)magnetic 
instability is more complicated. Some approximate cal- 
culations in this direction are presented recently 

Nevertheless, the mixed phase or phase separation [3, 
[lo| which has been observed in recent experiments flllll^ 
is not included in our discussion. We have planned to 
give a more complete work including the important mixed 
phase 67j. 
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